Will the REF disadvantage interdisciplinary research? The inadvertent effects of journal rankings

A failure to engage in interdisciplinary work risks creating intellectual inbreeding and could push research away from socially complex issues. Ismael Rafols asks why there is a bias against interdisciplinary research, and why the REF will work to suppress an otherwise useful body of research

A avaliação da pesquisa baseada em periódicos marginaliza regiões como a América Latina e seus problemas mais relevantes

Muitos sistemas de avaliação adotam uma abordagem limitada de excelência, ao julgar o valor de um trabalho pelo periódico em que é publicado. Pesquisas recentes de Diego Chavarro, Ismael Ràfols e outros colegas, mostram como esses sistemas subestimam e prejudicam a produção de importantes questões sociais, econômicas e ambientais. Esses sistemas também refletem os preconceitos das bases de dados de citações, que se concentram principalmente na pesquisa escrita em inglês, publicada em revistas nos Estados Unidos e no oeste e norte da Europa. Além disso, os tópicos abrangidos por esses bancos de dados muitas vezes respondem mais aos interesses dos setores industriais do que aos das comunidades locais. É necessária uma avaliação mais abrangente e inclusiva da pesquisa para superar a marginalização contínua dos povos, línguas e disciplinas e promover a inclusão, e não o elitismo

Research assessments based on journal rankings systematically marginalise knowledge from certain regions and subjects

Many research evaluation systems continue to take a narrow view of excellence, judging the value of work based on the journal in which it is published. Recent research by Diego Chavarro, Ismael Ràfols and colleagues shows how such systems underestimate and prove detrimental to the production of research relevant to important social, economic, and environmental issues. These systems also reflect the biases of journal citation databases which focus heavily on English-language research from the USA and north and western Europe. Moreover, topics covered by these databases often relate to the interests of industrial stakeholders rather than those of local communities. More inclusive research assessments are needed to overcome the ongoing marginalisation of some peoples, languages, and disciplines and promote engagement rather than elitism

Journal-based research assessments marginalise regions like Latin America and the issues most relevant to them

Many research evaluation systems continue to take a narrow view of excellence, judging the value of work based on the journal in which it is published. Recent research by Diego Chavarro, Ismael Ràfols, and colleagues shows how such systems underestimate and prove detrimental to the production of research on important social, economic, and environmental issues. These systems also reflect the biases of journal ..

La evaluación de la investigación basada en revistas margina a regiones como América Latina y sus temas más relevantes

Muchos sistemas de evaluación adoptan un enfoque limitado de excelencia, al juzgar el valor de un trabajo por la revista en que se publica. Una investigación reciente de Diego Chavarro, Ismael Ràfols y otros colegas muestra cómo tales sistemas subestiman y son perjudiciales en la producción de importantes temas sociales, económicos y ambientales. Estos sistemas también reflejan los sesgos de las bases de datos de citación, que se enfocan principalmente en investigaciones escritas en inglés publicadas en revistas de los Estados Unidos y el occidente y el norte de Europa. Además, los temas cubiertos por estas bases de datos responden con mucha frecuencia más a los intereses de sectores industriales que a los de las comunidades locales. Se requiere entonces una evaluación de la investigación más amplia e incluyente para superar la marginalización continua de pueblos, idiomas y disciplinas y para promover la inclusión y no el elitismo

A call for inclusive indicators that explore research activities in “peripheral” topics and developing countries

Science and Technology (S&T) systems all over the world are routinely monitored and assessed with indicators that were created to measure the natural sciences in developed countries. Ismael Ràfols and Jordi Molas-Gallart argue these indicators are often inappropriate in other contexts. They urge S&T analysts to create data and indicators that better reflect research activities and contributions in these “peripheral” spaces

How to make altmetrics useful in societal impact assessments: shifting from citation to interaction approaches

The suitability of altmetrics for use in assessments of societal impact has been questioned by certain recent studies. Ismael Ràfols, Nicolas Robinson-García and Thed N. van Leeuwen propose that, rather than mimicking citation-based approaches to scientific impact evaluation, assessments of societal impact should be aimed at learning rather than auditing, and focused on understanding the engagement approaches that lead to impact. When using altmetric data for societal impact assessment, greater value might be derived from adopting “interaction approaches” to analyse engagement networks among researchers and stakeholders. Experimental analyses using data from Twitter are presented here to illustrate such an approach

Designing indicators for opening up evaluation: insights from research assessment

The use of indicators is generally associated with a reduction of perspectival diversity in evaluation that often facilitates making decisions along dominant framings – effectively closing down debate. In this chapter we will argue that while this is indeed often the case, indicators can also be used to help support more plural evaluation and foster more productively critical debate. In order to achieve this shift, it is necessary equally to change understandings, forms and uses of indicators in decision making. These shifts involve, first, broadening out the range of ‘inputs’ taken into account; and second, opening up the ‘outputs’, in the sense of developing methodologies for indicator-based analyses to help in considering plural perspectives. In practice, this means a move towards more situated and participatory use of quantitative evidence in evaluation, a shift from universal indicators to contextualized indicating.Teaching and Teacher Learning (ICLON)Merit, Expertise and Measuremen